Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

993 cloud watch launch templates #999

Open
wants to merge 6 commits into
base: develop
Choose a base branch
from
Open

993 cloud watch launch templates #999

wants to merge 6 commits into from

Conversation

philipjyoon
Copy link
Contributor

Purpose

  • Cleaned up the aws cloud agent log collection list so that the list contains only the file paths actually relevant to the specific verdi worker. Prior to this change, every verdi worker aws cloud agent was looking through ~40 file paths and incurred up to 200% CPU utilization.

Proposed Changes

  • Parameterized ASG launch template user data creation so that it only lists files paths relevant to the verdi worker type.

Issues

Testing

  • Ran all SCIFLO and most query and download jobs. Verified that all logs were being collected into cloudwatch as before this change.
  • DISP-S1 SCIFLO jobs seems to run faster which is expected. Small sample size of 3 suggests 20% improvement in performance but will need to do a larger, more controlled study. Performing such study now.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@chrisjrd chrisjrd Awaiting requested review from chrisjrd

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
1 participant
@philipjyoon